Reliability and Fault Tolerance in Trust

The ubiquity of information systems has made correct and reliable operation of critical systems indispensable. The trustworthiness of digital systems is increasingly dependent on the trustworthiness of the software. While hardware trustworthiness is by no means a solved problem, systemwide problems are increasingly blamed on poorly tested, defective software. System trustworthiness is therefore a combination of several key software attributes: reliability, safety, security, availability, performance, fault-tolerance, and privacy. Some of these attributes can be directly measured, some cannot. For example performance and availability can be numerically measured; safety and security cannot. Further, several of these attributes may conflict, such as security and performance. Therefore to demonstrate that the software of a system can be trusted, it requires a combination of qualitative arguments concerning the level achieved for some attributes in combination with the numerical (quantitative) scores measured for others. In order to understand the trustworthiness and security of a software system, we first need to understand its reliability and fault tolerance. Therefore I will propose here the need for new thinking in these two areas. To begin, software reliability theory is one of industry’s seminal approaches for predicting the likelihood of software field failures. Unfortunately, the assumptions that software reliability measurement models make do not address the complexities of most software, resulting in far less adoption of theory into practice than is possible. Even though reliability models are quantitative, the industry only uses these results qualitatively. An example of this use is testing. When the mean time to failure (MTTF) falls below X according to a particular reliability model, testing stops, but developers and users cannot assume that the software will always behave with an MTTF less than X in the field. The rationale for researching a new theory of software reliability is clear: current software reliability theories do not scale to the types of large-scale heterogeneous systems that are being fielded [1],[3]. Existing software reliability theories work more accurately in telecommunications and aerospace because telecom and aerospace software works in embedded environments, and in many ways is indistinguishable from the hardware on which it resides. In other disciplines, such as the production of commercial, shrinkwrap software, quality has historically been an add-on, of lesser market value than feature richness or short release cycles. Further, there are dozens of underlying variables concerning different hardware and computer configurations that are not included in the traditional definition of the application’s operational profile that cause the same application to have numerous different reliabilities depending on its environment. These variables are not properly included in the current definition of operational profile. Therefore new research should attempt to create a software reliability theory for all software, particularly COTS software and “componentware.” It should also examine the use of time in existing models, and look at alternatives parameters such as fault detecting ability offered by the operational profile and the complexity of the code. And it should also examine how the current definition of an operational profile can be expanded to more accurately define the input space of non-embedded software. Other benefits of such research include the potential to create a composability theory for component-based systems, by reducing the current problems of incompatible assumptions between interconnected components. Next, I will argue that true fault tolerance cannot be achieved without specialized testing that involves fault insertion techniques at the interface level. We are proposing new thinking that investigates the adoption of methodologies to assess the interoperability of components [2], without assuming that the code within the components is accessible. Such thinking contains two key components: (1) the ability to corrupt internal states during test execution, and (2) the required assertions to determine the impact of the corruptions as well as the rate of